“Are We There Yet?” Visual Tracking of Visitors Under Variable-Lighting Conditions for a Responsive Audio Art Installation

For a responsive audio art installation in a skylit atrium, we developed a single-camera statistical segmentation and tracking algorithm. The algorithm combines statistical background image estimation, per-pixel Bayesian segmentation, and an approximate solution to the multi-target tracking problem using a bank of Kalman filters and GaleShapley matching. A heuristic confidence model enables selective filtering of tracks based on dynamic data. Experiments suggest that our algorithm improves recall and F2-score over existing methods in OpenCV 2.1. We also find that feedback between the tracking and the segmentation systems improves recall and F2-score. The system operated effectively for 5-8 hours per day for 4 months. Source code and sample data is open source and available in OpenCV. Fig. 1. The skylit atrium that was the site for this installation at the Contemporary Jewish museum in San Francisco, CA from April July 2011.